API reference¶
API reference for TTP module.
-
class
ttp.
ttp
(data='', template='', log_level='WARNING', log_file=None, base_path='', vars={})¶ Template Text Parser main class to load data, templates, lookups, variables and dispatch data to parser object to parse in single or multiple processes, construct final results and run outputs.
Parameters
data
file object or OS path to text file or directory with text files with data to parsetemplate
file object or OS path to text file with templatebase_path
(str) base OS path prefix to load data from for template’s inputslog_level
(str) level of logging “DEBUG”, “INFO”, “WARNING”, “ERROR”, “CRITICAL”log_file
(str) path where to save log filevars
dictionary of variables to make available to ttp parser
Example:
from ttp import ttp parser = ttp(data="/os/path/to/data/dir/", template="/os/path/to/template.txt") parser.parse() result = parser.result(format="json") print(result[0])
-
add_function
(fun, scope, name=None, add_ttp=False)¶ Method to add custom function in
_ttp_
dictionary. Function can be referenced in template depending on scope.Parameters
fun
- function referencescope
- scope to add function toname
- optional, name to use within templates, by default equal to function__name__
add_ttp
- boolean, on True will add_ttp_
dictionary in function’s global scope
scope options
match
- used for match variablesgroup
- used for groupsinput
- used for inputsoutput
- used for outputsreturners
- used for output returnersformatters
- used for output returnersvariable
- used as template variable gettermacro
- used as macro function
Warning
add_function should be called before template loaded in parser
Custom functions should use first argument to hold data to process, additional args and kwargs will be supplied to function if provided in template.
TTP passes output tag attributes to
returner
andformatter
functions, attributes need to be unpacked using, for instance,**kwargs
.For
template variable
getters functions first argument supplied is an input text data, second argument is datum name, equal to filename if loaded from fileFunction return content differ depending on scope:
match
- must return tuple of two itemsgroup
- must return tuple of two itemsinput
- must return tuple of two itemsoutput
- must return single element containing processing resultsreturners
- not returns expectedformatters
- must return single element containing processing resultsvariable
- must return single element to assign to variablemacro
- can return processing results, True or False
For
match
,group
andinput
functions TTP expects in return tuple of two elements where first element should contain processing results, second element can be True, False, None or dictionary. Second item can influence processing logic following these rules:- if second item is False - results invalidated and discarded
- if second item is True or None - first item replaces originally supplied data, processing continues
- if second item is dictionary - supported by
match
scope only, dictionary merged with results
Example:
def group_cust_fun(data, *args, **kwargs): if kwargs.get("upper") == True: data["description"] = data["description"].upper() return data, None template = ''' <input load="text"> interface Lo1 description this interface has description ip address 1.1.1.1 32 </input> <group myFun="upper=True"> interface {{ interface }} description {{ description | ORPHRASE }} ip address {{ ip }} {{ mask }} </group> ''' parser = ttp() parser.add_function(group_cust_fun, scope="group", name="myFun") parser.add_template(template) parser.parse()
-
add_input
(data, input_name='Default_Input', template_name='_root_template_', groups=['all'])¶ Method to load data to be parsed. Data associated with certain input of
input_name
and template oftemplate_name
.Warning
add_input
should be called only after templates addedParameters
data
text data or OS path to text file or directory with text files with data to parse. Also can be structured data - list or dictionary - will be passed to input as is, so that it can be pre-processed using input macro function(s)input_name
(str) name of the input to put data in, default is Default_Inputgroups
(list) list of group names to use to parse this input datatemplate_name
(str) name of the template to add input for
-
add_lookup
(name, text_data='', include=None, load='python', key=None)¶ Method to add lookup table data to all templates loaded so far. Lookup is a text representation of structure that can be loaded into python dictionary using one of the available loaders - python, csv, ini, yaml, json.
Parameters
name
(str) name to assign this lookup table for referencingtext_data
(str) text to load lookup table/dictionary frominclude
(str) absolute or relative /os/path/to/lookup/table/file.txtload
(str) name of TTP loader to use to load table datakey
(str) specify key column for csv loader to construct dictionary
include
can accept relative OS path - relative to the directory where TTP will be invoked either using CLI tool or as a module
-
add_template
(template, template_name='_root_template_', filters=[])¶ Method to load TTP templates into the parser.
Parameters
template
file object or OS path to text file with templatetemplate_name
(str) name of the templatefilters
(list) list of templates’ names to load,
filters
attribute allow to filter the list of template names that should be loaded. Checks done against child templates as well. For templates specified in filter list, groups/macro/inputs/etc. will not be loaded and no results produced.
-
add_vars
(vars)¶ Method to add variables to ttp and its templates to reference during parsing
Parameters
vars
dictionary of variables to make available to ttp parser
-
clear_input
(template_name='_all_')¶ Method to delete all input data for all or some templates, can be used prior to adding new set of data to parse with same templates, instead of re-initializing ttp object.
Parameters
template_name
(str) name of the template to clear input for, clears for all templates by default
-
clear_result
(templates=[])¶ Method to clear parsing results for templates.
Parameters
templates
(list or str) - name of template(s) to clear results for, if not provided will clear results for all templates.
-
get_input_load
()¶ Method to retrieve input tag text load. Using input
load
attribute, text data can be loaded into python structure using one of the supported loaders, for instance if text data structured using YAML, YAML loader can be used to produce python native structure, that structure will be returned by this method.Primary use case is to specify parameters within TTP input that can be used by other applications/scrips.
Returns
Dictionary of {“template_name”: {“input_name”: “input load data”}} across all templates, where input_name set to input name attribute value, by default it is “Default_Input”, and template_name set to name of the template, by default it is “_root_template_”
Warning
inputs load can override one another if combination of template_name and input_name is not unique.
-
parse
(one=False, multi=False)¶ Method to parse data with templates.
Parameters
one
(boolean) if set to True will run parsing in single processmulti
(boolean) if set to True will run parsing in multiprocess
By default one and multi set to False and TTP will run parsing following below rules:
- if one or multi set to True run in one or multi process
- if overall data size is less then 5Mbyte, use single process
- if overall data size is more then 5Mbytes, use multiprocess
In addition to 3 TTP will check if number of input data items more then 1, if so multiple processes will be used and one process otherwise.
-
result
(templates=[], structure='list', **kwargs)¶ Method to get parsing results, supports basic filtering based on templates’ names, results can be formatted and returned to specified returner.
Parameters
templates
(list or str) names of the templates to return results forstructure
(str) structure type, valid values -list
,dictionary
orflat_list
kwargs - can contain any attributes supported by output tags, for instance:
format
(str) output formatter name - yaml, json, raw, pprint, csv, table, tabulatefunctions
(str) reference output functions to run results through
Example:
from ttp import ttp parser = ttp(data="/os/path/to/data/dir/", template="/os/path/to/template.txt") parser.parse() json_result = parser.result(format="json")[0] yaml_result = parser.result(format="yaml")[0] print(json_result) print(yaml_result)
Returns
By default template results set to per_input and structure set to list, returns list such as:
[ [ template_1_input_1_results, template_1_input_2_results, ... template_1_input_N_results ], [ template_2_input_1_results, template_2_input_2_results, ... ]
If template results set to per_template and structure set to list, returns list such as:
[ [ template_1_input_1_2...N_joined_results ], [ template_2_input_1_2...N_joined_results ] ]
If template results set to per_input and structure set to dictionary, returns dictionary such as:
{ template_1_name: [ input_1_results, input_2_results, ... input_N_results ], template_2_name: [ input_1_results, input_2_results ], ... }
If template results set to per_template and structure set to dictionary, returns dictionary such as:
{ template_1_name: input_1_2...N_joined_results, template_2_name: input_1_2...N_joined_results }
If structure set to flat_list, results will be combined across all templates in a list of dictionaries. For instance, with structure set to list result might look like this:
[[[{'interface': 'Lo0', 'ip': '192.168.0.1', 'mask': '32'}, {'interface': 'Lo1', 'ip': '1.1.1.1', 'mask': '32'}], [{'interface': 'Lo2', 'ip': '2.2.2.2', 'mask': '32'}, {'interface': 'Lo3', 'ip': '3.3.3.3', 'mask': '32'}]]]
But with structure set to flat_list it will be flattened to this:
[{'interface': 'Lo0', 'ip': '192.168.0.1', 'mask': '32'}, {'interface': 'Lo1', 'ip': '1.1.1.1', 'mask': '32'}, {'interface': 'Lo2', 'ip': '2.2.2.2', 'mask': '32'}, {'interface': 'Lo3', 'ip': '3.3.3.3', 'mask': '32'}]
-
set_input
(data, input_name='Default_Input', template_name='_root_template_', groups=['all'])¶ Method to replace existing templates inputs data with new set of data. This method is alias to
clear_input
andadd_input
methods.Warning
set_input
should be called only after templates addedParameters
data
text data or OS path to text file or directory with text files with data to parse Also can be structured data - list or dictionary - will be passed to input as is, so that it can be pre-processed using input macro function(s)input_name
(str) name of the input to put data in, default is Default_Inputgroups
(list) list of group names to use to parse this input datatemplate_name
(str) name of the template to set input for